Agent Name Accuracy: 93.6%
Plugin Name Accuracy: 93.6%
Arguments Accuracy: 87.2%
Overall Accuracy: 87.2%
| Query | Response | Skill Type | Intents | Functions | Agent Name | Arguments | Overall |
|---|---|---|---|---|---|---|---|
| turn off washing machine | The washing machine has been turned off. | [ washingmachine_control ] | Single Intent |
Expected Function:
Agent:control_device_operation
Plugin:washingmachine_control
Arguments:{"operation": "start"}
Result:Washing machine stop.
Predicted Function:
Agent:control_device_operation
Plugin:washingmachine_control
Arguments:{"operation": "stop"}
Result:Washing machine stop.
|
✓ | ✗ | ✗ |
| start the washing machine | The washing machine has been started. | [ washingmachine_control ] | Single Intent |
Expected Function:
Agent:control_device_operation
Plugin:washingmachine_control
Arguments:{"operation": "start"}
Result:Washing machine start.
Predicted Function:
Agent:control_device_operation
Plugin:washingmachine_control
Arguments:{"operation": "start"}
Result:Washing machine start.
|
✓ | ✓ | ✓ |
| pause the washing machine | The washing machine has been paused. | [ washingmachine_control ] | Single Intent |
Expected Function:
Agent:control_device_operation
Plugin:washingmachine_control
Arguments:{"operation": "pause"}
Result:Washing machine pause.
Predicted Function:
Agent:control_device_operation
Plugin:washingmachine_control
Arguments:{"operation": "pause"}
Result:Washing machine pause.
|
✓ | ✓ | ✓ |
| resume the washing machine | The washing machine has been resumed. | [ washingmachine_control ] | Single Intent |
Expected Function:
Agent:control_device_operation
Plugin:washingmachine_control
Arguments:{"operation": "resume"}
Result:Washing machine resume.
Predicted Function:
Agent:control_device_operation
Plugin:washingmachine_control
Arguments:{"operation": "resume"}
Result:Washing machine resume.
|
✓ | ✓ | ✓ |
| stop the washing machine | The washing machine has been stopped. | [ washingmachine_control ] | Single Intent |
Expected Function:
Agent:control_device_operation
Plugin:washingmachine_control
Arguments:{"operation": "stop"}
Result:Washing machine stop.
Predicted Function:
Agent:control_device_operation
Plugin:washingmachine_control
Arguments:{"operation": "stop"}
Result:Washing machine stop.
|
✓ | ✓ | ✓ |
| set the washing machine to quick wash mode | The washing machine is now set to quick wash mode. | [ washingmachine_control ] | Single Intent |
Expected Function:
Agent:set_mode
Plugin:washingmachine_control
Arguments:{"mode": "quick wash"}
Result:Washing machine set to quick wash mode.
Predicted Function:
Agent:set_mode
Plugin:washingmachine_control
Arguments:{"mode": "quick wash"}
Result:Washing machine set to quick wash mode.
|
✓ | ✓ | ✓ |
| set the washing machine to delicate mode | The washing machine is now set to delicate mode. | [ washingmachine_control ] | Single Intent |
Expected Function:
Agent:set_mode
Plugin:washingmachine_control
Arguments:{"mode": "delicate"}
Result:Washing machine set to delicate mode.
Predicted Function:
Agent:set_mode
Plugin:washingmachine_control
Arguments:{"mode": "delicate"}
Result:Washing machine set to delicate mode.
|
✓ | ✓ | ✓ |
| set the washing machine to heavy load mode | The washing machine is now set to heavy load mode. | [ washingmachine_control ] | Single Intent |
Expected Function:
Agent:set_mode
Plugin:washingmachine_control
Arguments:{"mode": "heavy load"}
Result:Washing machine set to heavy load mode.
Predicted Function:
Agent:set_mode
Plugin:washingmachine_control
Arguments:{"mode": "heavy load"}
Result:Washing machine set to heavy load mode.
|
✓ | ✓ | ✓ |
| set the washing machine timer to 30 minutes | The washing machine timer has been set to30 minutes. | [ washingmachine_control ] | Single Intent |
Expected Function:
Agent:set_timer
Plugin:washingmachine_control
Arguments:{"time": "30 minutes"}
Result:Washing machine timer set to 30 minutes.
Predicted Function:
Agent:set_timer
Plugin:washingmachine_control
Arguments:{"time": "30 minutes"}
Result:Washing machine timer set to 30 minutes.
|
✓ | ✓ | ✓ |
| set the washing machine timer to 1 hour | The washing machine timer has been set to1 hour. | [ washingmachine_control ] | Single Intent |
Expected Function:
Agent:set_timer
Plugin:washingmachine_control
Arguments:{"time": "1 hour"}
Result:Washing machine timer set to 1 hour.
Predicted Function:
Agent:set_timer
Plugin:washingmachine_control
Arguments:{"time": "1 hour"}
Result:Washing machine timer set to 1 hour.
|
✓ | ✓ | ✓ |
| turn on the TV and increase the volume | The TV has been turned on, and the volume has been increased. | [ tv_control , tv_control ] | Multi Intent |
Expected Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Predicted Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Expected Function:
Agent:adjust_volume
Plugin:tv_control
Arguments:{"operation": "decrease"}
Result:TV volume increased.
Predicted Function:
Agent:adjust_volume
Plugin:tv_control
Arguments:{"operation": "increase"}
Result:TV volume increased.
|
✓ | ✗ | ✗ |
| turn on the TV and decrease the volume | The TV has been turned on, and the volume has been decreased. | [ tv_control , tv_control ] | Multi Intent |
Expected Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Predicted Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Expected Function:
Agent:adjust_volume
Plugin:tv_control
Arguments:{"operation": "decrease"}
Result:TV volume decreased.
Predicted Function:
Agent:adjust_volume
Plugin:tv_control
Arguments:{"operation": "decrease"}
Result:TV volume decreased.
|
✓ | ✓ | ✓ |
| turn on the TV and mute it | The TV has been turned on. Unfortunately, muting the TV is currently not available. However, I can decrease the volume if you'd like! | [ tv_control ] | Multi Intent |
Expected Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Predicted Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
|
✗ | ✗ | ✗ |
| turn on the TV and set the volume to 15 | Setting the TV to a specific volume level is not directly supported; I can only increase or decrease the volume. Would you like me to assist in adjusting the volume incrementally? | [ tv_control ] | Single Intent |
Expected Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Predicted Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
|
✓ | ✓ | ✓ |
| turn on the TV and switch to HDMI 1 | The TV has been turned on, and the input source has been switched to HDMI1. | [ tv_control , tv_control ] | Multi Intent |
Expected Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Predicted Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Expected Function:
Agent:switch_input_source
Plugin:tv_control
Arguments:{"input_source": "HDMI 1"}
Result:TV input switched to HDMI 1.
Predicted Function:
Agent:switch_input_source
Plugin:tv_control
Arguments:{"input_source": "HDMI 1"}
Result:TV input switched to HDMI 1.
|
✓ | ✓ | ✓ |
| turn on the TV and open Netflix | The TV has been turned on, and Netflix is now open. | [ tv_control , tv_control ] | Multi Intent |
Expected Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Predicted Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Expected Function:
Agent:open_application
Plugin:tv_control
Arguments:{"app_name": "Netflix"}
Result:Netflix opened on the TV.
Predicted Function:
Agent:open_application
Plugin:tv_control
Arguments:{"app_name": "Netflix"}
Result:Netflix opened on the TV.
|
✓ | ✓ | ✓ |
| turn on the TV and increase the brightness | The TV can be turned on, but brightness adjustments need a specific level set (e.g.,50%,70%). Do you want me to turn on the TV and set a particular brightness level? | [ tv_control ] | Multi Intent |
Expected Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Predicted Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
|
✗ | ✗ | ✗ |
| turn on the TV and set the channel to 101 | The TV has been turned on, and the channel is set to101. | [ tv_control , tv_control ] | Multi Intent |
Expected Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Predicted Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Expected Function:
Agent:set_channel
Plugin:tv_control
Arguments:{"channel": "101"}
Result:TV channel set to 101.
Predicted Function:
Agent:set_channel
Plugin:tv_control
Arguments:{"channel": "101"}
Result:TV channel set to 101.
|
✓ | ✓ | ✓ |
| turn on the TV and enable subtitles | The TV has been turned on, and subtitles have been enabled. | [ tv_control , tv_control ] | Multi Intent |
Expected Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Predicted Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Expected Function:
Agent:enable_subtitles
Plugin:tv_control
Arguments:{}
Result:Subtitles enabled on the TV.
Predicted Function:
Agent:enable_subtitles
Plugin:tv_control
Arguments:null
Result:Subtitles enabled on the TV.
|
✓ | ✗ | ✗ |
| turn on the TV and switch to HDMI 2 | The TV has been turned on, and the input source has been switched to HDMI2. | [ tv_control , tv_control ] | Multi Intent |
Expected Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Predicted Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Expected Function:
Agent:switch_input_source
Plugin:tv_control
Arguments:{"input_source": "HDMI 2"}
Result:TV input switched to HDMI 2.
Predicted Function:
Agent:switch_input_source
Plugin:tv_control
Arguments:{"input_source": "HDMI 2"}
Result:TV input switched to HDMI 2.
|
✓ | ✓ | ✓ |
| turn on the TV and open YouTube | The TV has been turned on, and YouTube is now open. | [ tv_control , tv_control ] | Multi Intent |
Expected Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Predicted Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "on"}
Result:TV turned on.
Expected Function:
Agent:open_application
Plugin:tv_control
Arguments:{"app_name": "YouTube"}
Result:YouTube opened on the TV.
Predicted Function:
Agent:open_application
Plugin:tv_control
Arguments:{"app_name": "YouTube"}
Result:YouTube opened on the TV.
|
✓ | ✓ | ✓ |
| turn off the TV | The TV has been turned off. | [ tv_control ] | Single Intent |
Expected Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "off"}
Result:TV turned off.
Predicted Function:
Agent:control_device_operation
Plugin:tv_control
Arguments:{"operation": "off"}
Result:TV turned off.
|
✓ | ✓ | ✓ |
| increase the TV volume | The TV volume has been increased. | [ tv_control ] | Single Intent |
Expected Function:
Agent:adjust_volume
Plugin:tv_control
Arguments:{"operation": "increase"}
Result:TV volume increased.
Predicted Function:
Agent:adjust_volume
Plugin:tv_control
Arguments:{"operation": "increase"}
Result:TV volume increased.
|
✓ | ✓ | ✓ |
| decrease the TV volume | The TV volume has been decreased. | [ tv_control ] | Single Intent |
Expected Function:
Agent:adjust_volume
Plugin:tv_control
Arguments:{"operation": "decrease"}
Result:TV volume decreased.
Predicted Function:
Agent:adjust_volume
Plugin:tv_control
Arguments:{"operation": "decrease"}
Result:TV volume decreased.
|
✓ | ✓ | ✓ |
| set the air conditioner to 24 degrees | The air conditioner temperature has been set to24 degrees. | [ ac_control ] | Single Intent |
Expected Function:
Agent:set_temperature
Plugin:ac_control
Arguments:{"temperature": "24"}
Result:Air conditioner temperature set to 24 degrees.
Predicted Function:
Agent:set_temperature
Plugin:ac_control
Arguments:{"temperature": "24"}
Result:Air conditioner temperature set to 24 degrees.
|
✓ | ✓ | ✓ |
| turn off the air conditioner | The air conditioner has been turned off. | [ ac_control ] | Single Intent |
Expected Function:
Agent:control_device_operation
Plugin:ac_control
Arguments:{"operation": "off"}
Result:Air conditioner turned off.
Predicted Function:
Agent:control_device_operation
Plugin:ac_control
Arguments:{"operation": "off"}
Result:Air conditioner turned off.
|
✓ | ✓ | ✓ |
| set the air conditioner to cool mode | The air conditioner is now set to cool mode. | [ ac_control ] | Single Intent |
Expected Function:
Agent:set_mode
Plugin:ac_control
Arguments:{"mode": "cool"}
Result:Air conditioner set to cool mode.
Predicted Function:
Agent:set_mode
Plugin:ac_control
Arguments:{"mode": "cool"}
Result:Air conditioner set to cool mode.
|
✓ | ✓ | ✓ |
| increase the air conditioner temperature to 26 degrees | The air conditioner temperature has been increased to26 degrees. | [ ac_control ] | Single Intent |
Expected Function:
Agent:set_temperature
Plugin:ac_control
Arguments:{"temperature": "26"}
Result:Air conditioner temperature set to 26 degrees.
Predicted Function:
Agent:set_temperature
Plugin:ac_control
Arguments:{"temperature": "26"}
Result:Air conditioner temperature set to 26 degrees.
|
✓ | ✓ | ✓ |
| decrease the air conditioner temperature to 22 degrees | The air conditioner temperature has been decreased to22 degrees. | [ ac_control ] | Single Intent |
Expected Function:
Agent:set_temperature
Plugin:ac_control
Arguments:{"temperature": "22"}
Result:Air conditioner temperature set to 22 degrees.
Predicted Function:
Agent:set_temperature
Plugin:ac_control
Arguments:{"temperature": "22"}
Result:Air conditioner temperature set to 22 degrees.
|
✓ | ✓ | ✓ |
| turn on the air conditioner | The air conditioner has been turned on. | [ ac_control ] | Single Intent |
Expected Function:
Agent:control_device_operation
Plugin:ac_control
Arguments:{"operation": "on"}
Result:Air conditioner turned on.
Predicted Function:
Agent:control_device_operation
Plugin:ac_control
Arguments:{"operation": "on"}
Result:Air conditioner turned on.
|
✓ | ✓ | ✓ |
| set the air conditioner to fan mode | The air conditioner is now set to fan mode. | [ ac_control ] | Single Intent |
Expected Function:
Agent:set_mode
Plugin:ac_control
Arguments:{"mode": "fan"}
Result:Air conditioner set to fan mode.
Predicted Function:
Agent:set_mode
Plugin:ac_control
Arguments:{"mode": "fan"}
Result:Air conditioner set to fan mode.
|
✓ | ✓ | ✓ |
| set the air conditioner to dry mode | The air conditioner is now set to dry mode. | [ ac_control ] | Single Intent |
Expected Function:
Agent:set_mode
Plugin:ac_control
Arguments:{"mode": "dry"}
Result:Air conditioner set to dry mode.
Predicted Function:
Agent:set_mode
Plugin:ac_control
Arguments:{"mode": "dry"}
Result:Air conditioner set to dry mode.
|
✓ | ✓ | ✓ |
| set the air conditioner timer to 2 hours | The air conditioner timer has been set to2 hours. | [ ac_control ] | Single Intent |
Expected Function:
Agent:set_timer
Plugin:ac_control
Arguments:{"time": "2 hours"}
Result:Air conditioner timer set to 2 hours.
Predicted Function:
Agent:set_timer
Plugin:ac_control
Arguments:{"time": "2 hours"}
Result:Air conditioner timer set to 2 hours.
|
✓ | ✓ | ✓ |
| set the air conditioner timer to 4 hours | The air conditioner timer has been set to4 hours. | [ ac_control ] | Single Intent |
Expected Function:
Agent:set_timer
Plugin:ac_control
Arguments:{"time": "4 hours"}
Result:Air conditioner timer set to 4 hours.
Predicted Function:
Agent:set_timer
Plugin:ac_control
Arguments:{"time": "4 hours"}
Result:Air conditioner timer set to 4 hours.
|
✓ | ✓ | ✓ |
| start the dishwasher | The dishwasher has been started. | [ dishwasher_control ] | Single Intent |
Expected Function:
Agent:control_device_operation
Plugin:dishwasher_control
Arguments:{"operation": "start"}
Result:Dishwasher startd.
Predicted Function:
Agent:control_device_operation
Plugin:dishwasher_control
Arguments:{"operation": "start"}
Result:Dishwasher startd.
|
✓ | ✓ | ✓ |
| stop the dishwasher | The dishwasher has been stopped. | [ dishwasher_control ] | Single Intent |
Expected Function:
Agent:control_device_operation
Plugin:dishwasher_control
Arguments:{"operation": "stop"}
Result:Dishwasher stopd.
Predicted Function:
Agent:control_device_operation
Plugin:dishwasher_control
Arguments:{"operation": "stop"}
Result:Dishwasher stopd.
|
✓ | ✓ | ✓ |
| pause the dishwasher | The dishwasher has been paused. | [ dishwasher_control ] | Single Intent |
Expected Function:
Agent:control_device_operation
Plugin:dishwasher_control
Arguments:{"operation": "pause"}
Result:Dishwasher paused.
Predicted Function:
Agent:control_device_operation
Plugin:dishwasher_control
Arguments:{"operation": "pause"}
Result:Dishwasher paused.
|
✓ | ✓ | ✓ |
| resume the dishwasher | The dishwasher has been resumed. | [ dishwasher_control ] | Single Intent |
Expected Function:
Agent:control_device_operation
Plugin:dishwasher_control
Arguments:{"operation": "resume"}
Result:Dishwasher resumed.
Predicted Function:
Agent:control_device_operation
Plugin:dishwasher_control
Arguments:{"operation": "resume"}
Result:Dishwasher resumed.
|
✓ | ✓ | ✓ |
| set the dishwasher to eco mode | The dishwasher is now set to eco mode. | [ dishwasher_control ] | Single Intent |
Expected Function:
Agent:set_mode
Plugin:dishwasher_control
Arguments:{"mode": "eco"}
Result:Dishwasher set to eco mode.
Predicted Function:
Agent:set_mode
Plugin:dishwasher_control
Arguments:{"mode": "eco"}
Result:Dishwasher set to eco mode.
|
✓ | ✓ | ✓ |
| set the dishwasher to heavy wash mode | The dishwasher is now set to heavy wash mode. | [ dishwasher_control ] | Single Intent |
Expected Function:
Agent:set_mode
Plugin:dishwasher_control
Arguments:{"mode": "heavy wash"}
Result:Dishwasher set to heavy wash mode.
Predicted Function:
Agent:set_mode
Plugin:dishwasher_control
Arguments:{"mode": "heavy wash"}
Result:Dishwasher set to heavy wash mode.
|
✓ | ✓ | ✓ |
| set the dishwasher to quick wash mode | The dishwasher is now set to quick wash mode. | [ dishwasher_control ] | Single Intent |
Expected Function:
Agent:set_mode
Plugin:dishwasher_control
Arguments:{"mode": "quick wash"}
Result:Dishwasher set to quick wash mode.
Predicted Function:
Agent:set_mode
Plugin:dishwasher_control
Arguments:{"mode": "quick wash"}
Result:Dishwasher set to quick wash mode.
|
✓ | ✓ | ✓ |
| set the dishwasher timer to 1 hour | The dishwasher timer has been set to1 hour. | [ dishwasher_control ] | Single Intent |
Expected Function:
Agent:set_timer
Plugin:dishwasher_control
Arguments:{"time": "1 hour"}
Result:Dishwasher timer set to 1 hour.
Predicted Function:
Agent:set_timer
Plugin:dishwasher_control
Arguments:{"time": "1 hour"}
Result:Dishwasher timer set to 1 hour.
|
✓ | ✓ | ✓ |
| set the dishwasher timer to 2 hours | The dishwasher timer has been set to2 hours. | [ dishwasher_control ] | Single Intent |
Expected Function:
Agent:set_timer
Plugin:dishwasher_control
Arguments:{"time": "2 hours"}
Result:Dishwasher timer set to 2 hours.
Predicted Function:
Agent:set_timer
Plugin:dishwasher_control
Arguments:{"time": "2 hours"}
Result:Dishwasher timer set to 2 hours.
|
✓ | ✓ | ✓ |
| turn on the dishwasher and start cleaning | The dishwasher has been turned on and started cleaning. | [ dishwasher_control ] | Multi Intent |
Expected Function:
Agent:control_device_operation
Plugin:dishwasher_control
Arguments:{"operation": "start"}
Result:Dishwasher startd.
Predicted Function:
Agent:control_device_operation
Plugin:dishwasher_control
Arguments:{"operation": "start"}
Result:Dishwasher startd.
|
✗ | ✗ | ✗ |
| turn off the dishwasher | The dishwasher has been turned off. | [ dishwasher_control ] | Single Intent |
Expected Function:
Agent:control_device_operation
Plugin:dishwasher_control
Arguments:{"operation": "stop"}
Result:Dishwasher stopd.
Predicted Function:
Agent:control_device_operation
Plugin:dishwasher_control
Arguments:{"operation": "stop"}
Result:Dishwasher stopd.
|
✓ | ✓ | ✓ |
| set the dishwasher to sanitize mode | The dishwasher is now set to sanitize mode. | [ dishwasher_control ] | Single Intent |
Expected Function:
Agent:set_mode
Plugin:dishwasher_control
Arguments:{"mode": "sanitize"}
Result:Dishwasher set to sanitize mode.
Predicted Function:
Agent:set_mode
Plugin:dishwasher_control
Arguments:{"mode": "sanitize"}
Result:Dishwasher set to sanitize mode.
|
✓ | ✓ | ✓ |
| set the dishwasher to rinse only mode | The dishwasher is now set to rinse-only mode. | [ dishwasher_control ] | Single Intent |
Expected Function:
Agent:set_mode
Plugin:dishwasher_control
Arguments:{"mode": "rinse only"}
Result:Dishwasher set to rinse only mode.
Predicted Function:
Agent:set_mode
Plugin:dishwasher_control
Arguments:{"mode": "rinse only"}
Result:Dishwasher set to rinse only mode.
|
✓ | ✓ | ✓ |